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ABSTRACT ' 

. Errors ma'de in' the allocation of federal funds to . 
institutions for the three campus-based a-id programs specified- in 
Title IV of the Higher Education Act are discussed. Funds are 
allocated directly to participating colleges, which in turn award the 
money to students, for the following programs: Supplemental Education 
Opportunity Grants, College WorJfr-Study, and National Direct Student 
Loans. Attention is focused on misallocation in the funds caused -by 
misreporting of institutional enrollments on the form entitled the 
"Fiscal-Operations Report and Application to Participate" . (FISAP) . 
Error definitions and data collection problems are considered, and , 
formulas for estimating need and allocation error are included. 
Potential errors. were 'identified by comparing information submitted 
by about 250 colleges on the FISAP to comparable data" submitted on, 
other documents, particularly the Higher Education General 
Information Surve^, and FISAPs for the previous year. 'For each 
institution with a FISAP enrollment, error, recalculating the 4 
allocation for each ^ampus-based program revealed a small overall 
change, with many . institutions being unaffected. Information is also 
provided on the calculation of the- national error levels, and 
recommendations for the U.S. Department of Education are included. 
<sw) 7 * 
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MISALL0CATI6N OF CAMPUS-BASED • . - 

PROGRAM FUNDS RESULTING FROM FJ5AP ERROR : 

1.0 ^MTRODUCTJOfi 

• Under Title IV of the. Higher Education Act, money for three Campus-Based 
progranu-Supplemental Education Opportunity Grants (SEofe), College Work-Study 
(CW-S), and National Direct Student Loans (NDSL) is allocated directly to 
participating institutions of higher education. These institutions, in turn, award the 
money to individual students to help them pay the cost of their pogtsecondary 
education. As part of the Title IV Quality Control Study, Advanced Technology, Inc.i 
of Reston, Virginia, was contracted to study the extent to which errors are made in 
the institutional 'allocation of funds for each program. This paper is the last in a. series 

of reports resulting from that task. 

* - . ♦ " * 

In this section of the report we summarize the allocation processed the purpose 
of the reporr and list the other reports which have been results of the <FISAP error 
•task. In sectil| 2.0 we define the error we are evaluating, describe the problems 'we 
encountered in collecting the necessary data, and specify the limitations of the report. 
In section 3.0 we describe our estimation methods, present and analyze the error 
estimates, and recommend steps to reduce the error. - 
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l ml Xhc Allocation Process 

For each fiscal year, Congress appropriates money speci-ficaily ^or each of the 
three Campus-Based programs. Unlike some other student .aid programs, Campus- 
Based aid is not an entitlement; neither individuals nor institutions have any statutory 
right to aid. Therefore, institutions must apply for funding annually jo the U.S. 
Department of Education. ; They do this by submitting the "Fiscal-Operations Report 
and Application to Participate (FISAP)" in the Title IV programs. This form both 
provides the information needed to allocate the next year?s appropriation and serves as 
a report on the use of the previous year's funds. (For instance, the reports submitted 
by October 1984 included applications for funds to award to students in the I9S5-S6 
academic year -and" repbrts of funds spent in the 19S3-S4 academic year.} This paper 
concerns only the effects of errors' in the parts of the FISAP used to^ determine how 
much money each school will receive. 

* 
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The atlc&ajwn of funds to individual institutions is a four-step process. First, 
most of the national appropriation is divided among the states. Second, institutions 
that have been particlraiting in the program receive "conditional guarantees" based on 
their funding ieveis irithe-^TSsSQ program year, [i the totaj of the conditional 
guarantees for ail institutions in k State exceeds that state's allocation, institutional 
allocations are reduced proportionately and the allocation process for institutions in 
the state effectively ends. Institutions may "also receive less than the conditional 
guarantee under some other circumstances, such as "a high default rate in the N DSL 
program. If funds remain from the state allocation after ail conditional guarantees in 
the state have been fulfilled, the excess is divided among the institutions so. tha$ their 
allocations will be more proportionate to their aggregate student need. Two different 
formulas are used to determine the aggregate need fof aid* of the students at an 
institution, one to allocate SEOG sfunds and- one for the CW-S and NDSb programs. 
The final step is to aliqcate a national pool, of funds among institutions to bring their 
allocations closer to the proportions their, needs bear to the total of ail institutional 
needs. However, in the year we are examining, 1983-84, there was no such national 
"fair share" money available. ; 

- ^ • * ■ ' 

• Funds for each of the*three ^programs are allocated separately; therefore, there 

are three different allocation formulas. These formulas differ" because different 

amounts of money are appropriated for each program and because each program has 

unique features which must be included^ For instance, the NDSL formula takes into 

account institutional collections and default rates, which do not exist in the SEOG and 

CW-S programs. 11 



\ Only a few numbers from the FISAP are included in the allocation formulas. 

1 V 

Total institutional income from tuition and fees is divided by the number of students 
«to determine average tuition and fees separately for undergraduates and graduate 
students. Avenge tuition and fees is used with the number of eligible aid applicants in 
each of 60 income and dependency- status categories to determine total self -Help need- 
Institutions can err l^arfsunderstanding exactly whidrdata are required for the FISAP, 
or by reporting^ the wrong numbers through transcription error.' Any number derived 
through calculation using several FISAP data elements will be in error, if any one of 
the elements is reported incorrectly. 
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1.2 Rationale of This Report 

v 

* * 

This report concerns only misaliocation of Campus-Based funds caused by 

• misreporting of institutional enrollments on the FISAP. . We discuss error definitions, 
data collection problems, and* the limitations of this^report in section 2.Q. In section 

3.0 we explain our methods of estimating allocation error, present the estimates, * ' 
analyze 4 the reasons f for the errors, and suggest ways fo reduce the error rates.' Ail 
m calculations assume that other factors in the ailocation'process remained constant.' - 

Limitations on the resources available for data collection for this fask 
determined how we would collect and analyze the data. Stage I of the Tjtie IV- Quality 
Gontroi study examined a national sample of 275 institutions participating in the 
Campus-Based programs. Before our data^ collectors visited these schools, we 
compared elements of their FISAP's to comparable data" on other documents they had 
also submitted, principally the' Higher Education General -Information Survey (HEGIS) 
and FISAP's for the previous year. Only when we found inconsistencies between .the 

• FISAP and HEGIS data for the" same year or substantial cross-year changes did our- 
data collectors investigate the FISAP data during their visits; their inquiries were 

- limited to the problematic items. Therefore'; we made no attempt to investigate 
internal institutional errors that were reported consistently on different Federal 
forms. t . _ - 



1.3 Other Reports on .Error in the FISAP Proc 

This report is the last, in a series of FISAP reports submitted to the Division of 
Quality; Assurance, Office of Student Financial Assistance,, U.S; Department of 
Education, under the Title IV Quality Control Study, .contract number 300-8^-0020. 
The earlier reports were: t . 

• Analyses»of Error Associated with the Application and Allocation Aspects 
of the Campus-Based Programs (February 27, 1984) 

• Analysis of Error Associated with .the Application and Allocation Aspects 
o* the Campus-Based Programs; Results from Initial Data Collection (May 
19S5) * , — \ 

• \ 

• Improving Quality in the Application Portion of the Processing' 1 Fiscal 
Operation s Report and Application to Participate; An Approach to 
Developing and Refining Edit Checks (August 1984) ~ 



i 
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• ' Analysis of Error Associated with the Application and Allocation Aspects 
■ of the Campus-Based Program: Plan, for Remaining QC IV Stage I Analysis 
(October X9S4) : / 1 — 



The preliminary report submitted on October 5, j98» (Erro>^h tne Campus-Based , 
Program a nd GSL^ Certification ' Process: Flash Report) also included a chaptejr, 
"PreHminary 1 AnaJydSs of Error Associated with the Application Aspects of the 
' Campus.-Based Programs." . . • 



2.0 DEFINITIONS, PROBLEMS, AND LIMITATIONS 

There is a large number of -theoretical or possible errors wjthin the FISAP 
process which could lead to misaiiocation of Campus-Based funds to Institutions. 
However, data collection problems and other (imitations of this study restricted its 
focus to a subset of these errors. Data collection problems and potential errors have 
been described more fully in .previous reports, especially the May 1984 paper, and are ' 
only summarized here- • ' . 



2.1 Definitions of F1SAP Errors^" * * 




Errors affecting^tha^iiocation of Campus-Based funds can be made by the 
institution, ED, and processing contractors.' Institutional errors can be divided into 
conceptual^ errors and transcription, errors. Staff at the institution responsible for 
•compiling the data may misunderstand ED's requirements and definitions ant may, 
therefore, include students or funds *iich should be excluded or omit students or funds 
w^ich should be included. Institutions with branches or several campuses may be 
confused about whether to combiner data from them or 'report on them separately. 
Often various offices contribute data; the^ may misunderstand ^even if the office 
which actually compiles and submits ^the report has the correct definitions. Once an" 
institution has compiled the data for the FISAP— correctly or incorrectly— it can still 
J make ^arithmatic or transcription errors when filling out the FISAP form itself. 
Moreover, after the school sends in its FISAP, ED or its processing contractor can tose 
or misfile it, enter data 'incorrectly onto magnetic' tap©; or make errors in editing tne 
data or calculating the institutional allocations* 

Most (Of these errors could not be detected with the resources'and methodology 
of this study. Internal institutional errors, for instance, could only have oeeri found by 



BEST COPY |VAIi AP' f 




detailed, comprehensive audits of institutional records. Jbetection of data-processing 
errors wouid have required item-by-il&m comparisons of the FISAP's as submitted to 
the data entry contractor with machinf-readable output. Therefore, for this study we 
considered differences between FISAP entries and comparable items on other reports 
to be potential errors, to be investigated during visits to the institutions, which were 
intended pricipaiiy to collect other .data. "Other reports" included forms submitted in 
response to ^e Higher Edu^tioivGenerai Information Survey (HEQf»), reports- on 
expenditures of aid under the Pell Gran^>«^zram, and FISAPs for the previous year.^ 

This approach produced a total^ of 83 potential afrors or. "discrepancies," ' as 
shown in Table 1 (a slight revision of Table 1 in the Octoblr 198* report). We lotjnd.aii 
but 3 of the enrollment discrepancies £y comparing FISAP and HEGIS data; we 
discovered 3 enrollment discrepancies and the other 23 discrepancies in tuition and 
fees receipts, Pell expenditures, and* income of eligible aid applicants by comparing 
this yeac^ata to the same items in the previous year. Most of these, discrepancies 
• were resolved during the institutional yisits, but 27 were not. Of these, four were 
based on cross-year data. Although they had notjseen resolved, ev^n large yearly- 
changes were not considered conclusive evidence of an error. On 'the other hand, in. 
the remaining cases >*f enrollment error, institutions clearly had not fallowed 
instructions to transfer Wto the FISAP specific items from their HEGIS reports. 
Therefore, the errors being analysed in this report consist of failure to report the 
correct enrollment data. 



c 



2.2 Data Collection Problems 

The number of errors which could bej analyzed for this report was limited by 
several practical data collection problems. • The results of some of these problems can, 
be seen in Table 2 in ^he difference between "Discrepancies Identified for Review" a&d 
■"Responses Received" and in the lour cases of "Discrepancies* rv'ot Resolved" which did 
not involve enrollment' data- Other problems prevented even the initial identification 
of discrepancies. . ' 

Problems presenting identification of discrepancies included missing data and 

differences in definitions. For some schools, the relevant alternative data source, 

such as one* of the HEGIS surveys, was not available. Moreover, HEGIS coverage is not 

i 

complete— many proprietary, private nonprofit, and regional vocationai-technicai 
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TABLE 1 

ERRORS ASSOCIATED WITH THE 
APPLICATION PORTION OF THE FISAPs 

SELECTED STATISTICS . 



\ * 

K \ 






Comparisons 


Pitereptticiet Reaoivcd 


Dtscrepancies Not Resolved 


* . 

Diicr eprcy itctM 

— l ■ , 


Number of 

Uwtilied * 
For Review 


* 

Number 


IVrccnt 


Crow- 
Source 


Crott- 
Yen 


JfcMbcr tarce^uge 


• 


BcroNO«fe 


fcuoiiJnem b ' * 


60 




SS«* 


50 




29 


54% 


23 r 




Tuition and Fees 


% 


$ 


100 


3 




5 




3 


11 


Pelt Expenditures 






75 

-f 


3 




6 - 




0 

* 


0 


*t . 
- iibgibic Applicant 
If k omc 








* 0 


6 

4 


5 


S3 




TOTAL * % 


r si 


72 


1 


56 


1* 


• • * 
• 5 


63 


27 


31 



'0* ■ 



on 3 institution* identified lor item discrcp^K^ rctoJutiott, 61 provided infor fruition on 70 discrepancies 
b ^ 
OJ total efifolliiieni discrepancies, all but tlkrc* wore gyrated by comparisons, to HfcCIS. 

< Tr^nroUment discrepancy lor one institution was both cr o«-sourte and cross-year- It was counted ^ one separate error, with o|e resolution. 
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\ schools do not file HEGIS forms. For these schools, there was either no alternative 
data source or a source on which the* data could have different definitions. 'Data also 
had to be considered missing when FISAf and - HEGIS were completed for different 

* reporting units, such as dif f erentycomblnations of schools, branches, or campuses of a 
university. Finally, FISAP's themselves were missing for 25 of the 275 institutions. 



Field data collection problems also hindered the analysis. Some institutions wer$ 
unable to respond to inquiries about discrepancies because the only knowledgeable 
staff or the necessary documents were not available at the time of the visit. A few of 
the 275 sampled institutions did not participate in the Title IV Quality Control Project 
for reasons unrelated to the study of FISAP error. ' 



2.3 Limitations of This Report 

' '• - . r : ■ ' 

* Because of the problems ^discussed in the two preceding sections; only limited use 
can be made otf the data we collected on FISAP allocation errors. In the next chapter 
we discuss how we estimated national error levels and present and analyze our- 
estimates. The estimate* are, however, restricted to the allocation consequences of 
just _one*of the several potential £ISAP errore: not reporting the same student 
enrollment data on the FISAP as on the HEGIS Fail Enrollment report. Since the 
estimates here deal with error attributable to only one of several variables in the 
allocation algorithm, they must be interpreted as partial estimates of tfte total amount 
of error in the allocation process. 



, The second caution' which must be emphasized is that the existence of- "error 11 in 
allocation of Gampus-Based funds does not at all imply that elimination of this error 
woXild lead, to any. cost savings for the Federal government. The allocation process is 
an iterative one designed to distribute all available funds to participating institutions 
on the basis of student need. Errors cause some institutions to receive more money to 
distribute to their students than they would otherwise while other institution^ get less. 
Elimination of this error would lead 'to a different distribution of funds, closer to the* 
policy intent which the regulations and allocation formulas express, but not to any 
reduction in expenditures. 
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3.0 FISAP ALLOCAQO£f ERROR . 

1 » •. . ' 

m T 

■Within the limitations described in the- preceding section, we recalculated- the 
allocation for each Campus-Based program 'to each sample institution with a FISAP 
enrollment error. The overall change was small, with many of the institutions being 
unaffected. .We also estimated the change in aggregate need 'resulting from . FISAP 
enroiirhent error. . 1 

* 

3.1 JJethods of Estimating Need and Allocation Error ' 

* 

We estimated allocation error By substituting the correct (HEGIS) ertmiraent 
data for the information submitted on the FISAP and following' the effects of thiL 
change through the allocation . formula for each school. We then' multiplied each 
allocation change «by the institution's weight and summed the weighted changes to 
'produce a national estimate. * - v - 



Enrollment -data are 4 not used in' the allocation formula directly; Rather,: 
institutions, also report -total income for tuition and fees, which ED divides by the 
number of students to find the average tuition and feesYaid. Therefore, if an 
institution reported fewer students on the FISAP than* on the HEGIS Fall Enrollment* 
form, correcting the. enrollment data upward corrected average- tuition ancTf^paid 
.downward; conversely, reducing the enrollment raised the average tuition' and fees. 
Any change in tuition and fees can affect the aggregate, student need on which' 
institutional allocations are based by changing the average student's cost. 

The 5EOG need formula for institutional is ^ ' » 

- ■ ~~ • \ 

N =» f.75(T.. + L)A. I - F. - P. * S -.25(1. ) 

1 v. iu . iuJ iu *i " iu . iu 

• where * . - * • 

N = Need • 1 . ' ' • 

■ ' ■ ' ' • , 

T - Average tuition and<fees 

.» • / -» . 

m L = Average living cost, set at $3200 for 1983-S4 . , 
A = Number of eligible aid applicants m 
<\ r = Aggregate expected family contribution 



/ njVT or-'* ■ ' •;; 'r\| f 

J Y' .'• ». t j, V ' \ i l - 
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P . = Pell expenditures 

• * 

.S = Expenditures of state aid 5 4 

**' . « > . * 

I = .Institutional aid expenditures , . ,(* 

-and u indicates elements applying to or derived from undergraduates only. Therefore, 
the formula foe need, adjusted for FJ5AP 'enrollment error, N'. (primes indicate" 
recalculated elements! is simojy " r 

*l = (.75(T iu * 320b)A i J - F. u . S . u - .25(1^)' 

Recalculating- self-help need for the CW-S and NDSL recalculations was more 
difficult. Self-help, need is the sum of the self-help needs in 15 income categories* of 
^independent and/tejjendent undergraduates and gradate students, or 60 categories for 
institutions with post-baccalaureate students. Within a category, the self-help need is 
the number of eligible aid applicants in the category multiplied by the lesser, of cost 
(average" tuition and fees plus the $3200 allowed for living costs) minus, a constant 
expected family contribution for aiY aid Applicants in the category; or by 25 percent of 
Cost.. However, categorical self-help need may never be k les's than zero. Therefore 
the formula for a self-helped need at institution J. for student type j_ (where ± is u for 
undergraduates or £ for graduates) in income group k wjith dependency status H where. I 
is.d for dependent students and e for independent students) is: ■ ^ 

N ijkl n i jki ( m ax'{0;minC(T. j + 3200 - E ^ ( . 25 (T. . + 3200) >}) 
where ' ' 

< 

n = Number of eligible aid applicants in category " • 

. E =^5tandard EFC for students in category „ - { ' 

Therefore, * * * *' '„ ' ' " 

N ijk'l " n. jkl [max(0;min{{T^ + 3200 - S ^ ] _) ; ( . 2!Tt* • , + 3200)-;}.) 



an*d 




* 1 jki .^ ki * . ^ 1 . ^ - . s 

aving calculated need change for the "sample institutions with error,- we could i 
e 'the total need'efror among all institutions particpating in the Campus-Based 
ms. We multiplied the change in each institution's .allocation by a weight equal 
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. to, th*ftj£|rse tff its selection- probability, 



• • Therefore, tojal need error fqr^sach program is 



and total absolute* need 4 error }s the sum of the weighted institutional errors, 
disregarding their signs, pr ■ - \' • 




* N. /TM. 
im im* | im 



where 



R im* \~ reiative need for institution Hn state m •: . 

« # 

To hold the rest of the state constant, we divided the revised need by the state need 
adjusted for the change in need of the institution under consideration. Therefore 
R! * N f /f(TN. ) 4- N ! - N 1 



Institutions also receive adjustments to their allocations because some institutions do 
not ge,t all the money to which they- were entitled according to their reiative need, 
because they were being penalized for excessive NQSL defaults or had not applied for 
their -full entitlements.;- Th^caicuiations of these' adjustments were not reflected in 
any of the allocation worksheets to which we had access. Therefore, we adjusted each 



\ 



ERIC "V v tf> -^V* \ .;V ' : ' 13 



-%M \. : i,vt BEST COPY AVAILABLE. 



5?' 



« ' . J- 



■Changing the need elements in the allocation formulas has a domino effect which ' 
fs to changes in many of the other elements. Relative state need, for instance, is 
the institution's need divided by, the sum of the needs for ail institutions in the state/* 

or • *' ■ .. . . 



revised allocation by the ratio between the institution's original allocation on the basis 
of its incorrect FI5AP enrollment report' (before adjustment) .and its actual original 
allocation (ai|er adjustment), or \ ' 

. • l,x 0,i lid/ 0,1 , • . 

where ♦ , • * \ . v < 

x • • • ' 4' « * . ■ " " - ' . 

••> X. s allocation for school i ' 

1 i 

Subscript 0 indicates the allocation before adjustment, and subscript 1 indicates |he 
ailpcationtaf ter adjustment. ' - ' 



To calculate total net error, Y, -for a specific Campus-Bas^ program, we 
multiplied the change in ea£h institution's allocation by its weight, or 

v - j(w i( x U - x 1# .,.) ... . 

* # , . -- 

' * ■ ■ • 

Wfe also calculated total gross error, the sum of the absolute allocation changes, or 



3.2 s Estimates of Meed and Allocation Error 



Table 2 shows our estimates of total error in need calculations, both net and 
absolute, arising from misreporting of enrollment on the FISAP. Net errors amount to 
only 1.2 percent of self-help need at institutions with error, and 17 percent of SEOG 
need at those institutions. Absolute error is S.S percent of self-help need, and 19.4 
percent of SEOG need. Because there was no national fair share money in any of the 
programs in 19S3-S4, ED did not calculate natidnai need totals; therefore, we cannot 
relate these need errors to total need. T • 



Table 3 shows our estimates of the total amourft of allocation error, both net and 
absolute, with the number of institutions whose allocations were affected by FISAP 
errors and the magnitude of the error in relation -to total allocations. Of the , 23 
schools .that still had unresolved discrepancies in their FISAP enrollment reports after 
our visits, we. could not find original FISAP's for 2. Therefore, we did not have the ' 
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Need Type 
SEOG Need 
Self -Help Need 



TABLE 2 
ESTIMATES OF NEED ERROR 



Number of * Total need Change ($ Thousands) 
Schools - Net * Absolute 



21 
21 



33,883 
2,529 



38,982 
IS, 598 



TABLE 3 

ESTIMATES OF ALLOCATION ERROR 



IMumber of 
Schools 



Total Allocation 



Relative Allocation 



Program 


With 
Error 


With 
Allocation. 
Change 


Change 
{$ Thousands) 




Change 
(Percent) 


Net 


Absolute 


Net 


Absolute 


cw-s 


21 a 


15 


-759 


1,222 


Q.H 


0.22 


NDSL 


18 ab 


13 


♦i,iG2 


2,106 


0.64 


U16* 


SEOG 


. 21 


S 


-311 


311 


0.08 


0.08 4 


Total 


60 


36 




3,639 




0.33 



, FI5 ^rt werp ""available for two institutions; therefore, need could not be recalculated 
for CW-S and NDSL. 

b. 



Three schools did.not participate in the-NDSL program. 
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.detailed breakdown of- eligible aid applicants by* income which "we needed to 
recalculate need. • „ 

• 

Although ail the instituions bein^ considered here had FISAP errors whictvcould 
have affectejftheir allocations, overall ohiy 60 percent of the allocations woula* have 
changed. Most of the institutions «ith errors but no # allocation change were in states 
in whictfthe conditional -guarantees totalled more than the state's allocation; 
therefore, there was no state fair share money to distribute on the basis of aggregate 
need at individual institutions. One school bad no. N DSL , allocation because of an 
excessive default rate; this situation was not changed by recalculation of the school's 
aggregate seli-help need. • , 

j. '. • ■ ■ . . 

Total allocation errors, as estimated on "the basis of the pilot Title IV quality 
control sttWy were small in both absolute and relative terms. The sum of the weighted 
values of ail allocation changes was negative for tne CW-S arid , SEOG programs, 
indicating that the schools in error got more money than they should have; correction 
of these errors would have freed money for the institutions which did not misreporfr 
their enrollments. In the NDSL program, the sum of allocation changes was positive, 
indicating that these schools were underfunded as a result of their FISAP errors. The 
total amount of money in error— counting both positive and negative changes as errors 
in the same direction, rather than letting overal locations oif set un^raliocations-was 
slightly over $1 million in the CW-S program, 'slightly over $2 milUon in the NDSL 
Program, and only about $300,000 in the SEOG program. In the three programs 
combined, only about $3,600,000 was misallocated as a result of FISAP enrollment 
error. 

t .■ ■ ■ - 

The right columns of Table 3 show that these reallocations involve only 
miniscuie amounts in relation to- the total allocations. The errors amounted to hardly 
rqpre than 1 percent of NDSL Federal capital contributions, but to less than $1 for 
every $450 of CW-S subsidies and to about $1 for every $1"250 in SEOG grants. 

3.3 Analysis of Allocation Error 

, Not oni y w ^s total allocation error small, but most allocations w,ere unaffected 
by, misreoorting of enrollments on the FISAP. None of the IS institutions for which 
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.we could calculate ailoction changes in ali three, programs hag* no change in all three, 
but "Exemplar State College" can serve as an example of the several which f\ 
changes in two programs. 1 Table k shows key allocation variables before and 
correction. - Exemplar reported 3,747 undergraduates on its FISAP but *,6U in the 
HEGIS Fail Enrollment survey. Since total tuition and fees remained the same, the 
error increased average tuition and fees per student by 23 percent, a' rather large 
difference t Because students are allowed $3200 per year for living expenses, the cost 
of education. at this low-cost school was only raised.it by k 'percent; however, this 
smai^change caused a 25 percent increase in SEOG need and a 5k percent increase in 
the self-help need used to calculate GW-S allotments. 'In both cases, the institution's 
"fair share" of the state allotment increased nearly proportionately. However, the 
sum of SEOG conditional guarantees for all schools in the state exceeded the state's 
SEOG allocation. Therefore, schools did not even get their full conditional guarantees 
' and there was no money left over to allocate on the basis^of need. Although 
Exemplar's allocation would have b«pen less than Its conditional guarantee using either 
enrollment datura, it would have received its guarantee if enough money had been 
available; thus its SEOG allocation would have been unchanged. The CW-S allocation- 
worked out only slightly differently. Although .there was money available to add to 
CW-5 conditional guarantees in the state, Exemplar had no share in this money 
because even with its need exaggerated by the underreporting of its enrollment, its 
conditional guarantee exceeded Its state fair share. . 

f ¥ 

S. 

,/ 

Exemplar State's NDSL allocation was affected, however. Like its 'relative-, state 
need for CW-S, its NDSL relative state need fell 'by 35 percent.- The result' of this 
change was a decrease in its overall lending ability shortfall, (line 20 of Table 3) by $9 
percent, as its condi tonal guarantee for Federal capital contributions, plus funds from 
collections and the required institution matching, now nearly equalled" the revised 
level-of-effort guarantee. Exemplar's share of the fundsavaiiabie in the state to meet 
these shortfalls fell the same S9 percent (line 22). Sinoe the conditional guarantee d^d 
not change, the sum of the conditional guarantee and funds from the state fair share 
pool decreased by only 69 percent, still a substantial drop (line. 23). 



*This is an actual case whose name we have disguised^ not a hypothetical one or 
a composite of several institutions. 
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13. 
14. 
15. 
16. 
17. 



18. 
19 
20. 
21. 
22. 



23. 



» • TABLE 4 

^ ALLOCATION EFFECTS OF FISAP ERROR 
AT EXEMPLAR STATE COLLEGE 

I 



1. Undergraduate Enrollment 

2. Average Tuition and Fees 

3. Cost of .Education 

SEOG 

4. Need 

5. Relative State Need 

6. State Fair Share 

7. Conditional Guarantee V 

8. Funds for State Shortfalls * 

9. *State Increase to Cond. Guar. 

10. Allocation 

- r 

11. Self-Help Need 



CW-S 

Relative State Need ' 
State. Fair Share 
Conditional Guarantee 
Funds for State Shortfalls 
State Increase to Cond. Guar. 
Allocation , 

NDSL . 
Conditional Guarantee— FCC 
Relative State Need 
Inst. State Shortfail-LOE 
Inst. Rel. State Shortfall— LOE 
State Increase to Adjusted 
Conditional Guarantee— FCC C 
Conditional Guarantee and State 
Increase— FCC (Allocation) 



Reported 
Data, 3 

3,747 
$ 832 
$ ,M32 



$1,179,622 
.0052245 
$ 101,378 \ 
$ 155,180 \ 
$ 0 
$ 0 J 
$ 151,021. 

$1,700,701 



.0066527 
$ 224,531 
$ 244,618 
$1,811,46$ 
$ 0 
$ 244,431 



'$ 9,229 
.0067129 

$ 87,293 
.0117140 

$ 30,578 

$ 39,807 



Best 
Data 15 ' 



$ 



$ 
$ 
$ 
$ 
$ 



4,614 
676 
3,876 



947,460 
.0042006 
. 81,510 
155,180 
0 
0 

151,021 



$1,107,250 



.0043413 
$ 146,457 
$ 244,618 
$1,811,463 
% 0 
$ 244,431 



$ 9,229 
.0043807 

$ 9,221 
.0012505 

$ 3,264 

$ 12,493 



Percent 
Change 

23 
23 
4 



25 
24 



54 



35 
53 



35 
89 



89 

69. 



3 

Data reported by institution on FISAP or used by ED to calculate allocation. 

b * 
Data that should have been reported by institution or would Have been used, bv 
-ED if fISAP had been correct. - - T 

r 

Federal capital contribution— the amount which the Federal government will 
contribute to add to the institution's- revolving fund. 

d • « « 

-Level of Effort— total new loans to be made by. the institution during the award 
year, from repayments of old loans, 8CC, and institutions matching funds (10 percent 
of FCC). . 
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Exemplar State bad a larger error in its original enrollment report than many of 
the 23 institutions being analyzed here. As the example demonstrates, such errors can 
be magnified greatly by the allocation formulas, but only if there is money in the state 
. fair share pool to add to conditional guarantees. 

v ■■ • . ■ • ' ' ■ " 

, Although other .effects of FISAP errors may be greater, the errors we have been 
„ able^to analyze here are so small that they should not be a subject of major concern. 
It is true that some institutions would get more or less mone^ frbm at least one of the 
Campus-Based programs if every institution reported the correct number on the 
FISAP. However, our, error estimates indicate that^the average absolute allocation 
error for schools with.FISAP enrollment errors was about $5,000 in the CW-S program, 
about $10,000 in NDSL Federal capital contributions, and about $1,200 in the SEOG 
program. Institutions which completed their FISAP's correctly would -be. affected 
much less, probably on the order of a few hundred dollars on average in the CW-S arid 

NDSL programs a few tens of dollars in the SEOG program. 

. -/ ' » 

3.*- Institution Justifications and Comments J - 

■ : • ' \ 

Since our data collection for this report included interviews with financial aid 
directors of sampled institutions at which they were asked to explain the discrepancies 
between FISAP and HE CIS enrollment data, we have a body of qualitative data which 
suggests some of the reasons for misrepdrting. At some institutions, the financial aid 
directors evidently did not agree that the enrollment data requested in the FISAP 
instructions accurately reflected their schools' "true" enrollments and so they 
submitted other figures. In other cases, the financial aid directors delegated the 
enrollment items of the FISAP to some other offices in the institutions' 
administrations, and did not know what figures their registrars, of fices of institutional 
research, or whatever had; entered. Our field staff ^5 not have time for foliowup 
interviews in these cases to try to find out wljy these other administrators had not 
followed the instructions. 

« 

3.5 Recommendations 

■ V £ : 

Although the enrollment reporting errors on the FISAP have only a slight effect 
on allocation of Campus-Sased funds, ED couidijake one* of three steps to reduce this 
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t par ticuia* problem: 



- • Clearer FISAP instructions \ 

• Cross-source editing. - I 

• Cross-source data collection 

Although th? FISAP instructions are clear enbugh about transferring HE CIS data, 
specking line numbers, it appears that some institutions do not realize tht 
importan(|^f repeating these numbers exactly. Therefore, they try to "improve" the ' 
accuracy of the data by* using other figures. An explanation of how the numbers are 
" used—that the enrollment data are used with the tuition and fees data to compute* • 
average tuition and fees—might reduce this source of error. In particular; pointing out 
the effects of adding to the numbers already reported to HEGIS— that this can reduce 
an institution's allocation— is likely to keep at least some institutions 'from losing' funds 
in ah honest but misguided effort to improve the accuracy of their FISAP's. 

. Routine cross-source editing— con<paring the enrollment data that has been 
entered onto magnetic tape with the comparable HEGIS items'— would repeal the same 
discrepancies for ail institutions that file both reports as we were able to uncover 
through a manual comparison of Jhe records for only 281 institutions. However, 
certain definitional problems could occur in comparing FISAP to HEGIS unless one of 
them were changed? see our report of August \9Zk, Improving Quality in the 
Applicatio n Portion of the Processing Fiscal Operations Report and Application to 
Participate; An Approa ch to Developing and Defining Edit Checks , page, 19. This, 
paper is attached as an appendix. ' 

Cross-source data collection would be a step beyond cross-source editing. Given 
that the data, on the FISAP discussed here are supposed to be the same as on HEGIS 
questionnaires, ED could take the data directly from HEGIS tapes for institutions 
whicfi are part of HEGIS. Institutions should still be asked to enter these data on the 
FISAP for two reasons; first, the data would then be available as a backup to HEGIS; 
second, instructions to omitiHEGIS data if the school is a HEGIS participant would 
probably lead to omis&pn by some schools which are. not part of HEGIS. 
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The second and third steps would require some adjustment of either HEGIS or the 
FISAP. The most practical adjustment would probably be to compute the Campus- 
Based allocations from revised or. updated HEGIS numbers rather than from the HEGIS 
data originally' submitted. TheoreticaUy, this step would produce , "better aiio'cations 
since the revised data should reflect institutional;enroliments more accurately than 
unrevised date, the allocation process would also have to be« changeft'so that 
allocations were made to the same units or subdivisions, such as individual colleges or 
campuses, that file their own HEGIS reports. * . 



/ 
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INTRODUCTION , 

Advanced Technology h,as conducted various analyses associated; with -the Fiscal 
Operations Report and Application to Participate (FISAP) in the ^Campus-Based 
student, aid programs. These analyses are part of the Title IV Quality Control study 
that will assess. for the Division of Quality Assurance (D9A), Office of r Student 
Financial Assistance <OSF A) the quality 6f .delivery of Campus-Based student aid funds . 
to students enrolled at Institutions participating in the three Campus-Based programs. 

^ . The first activity of the FISAP analysis involved data collection at the Depart- 
ment of Education (ED). 'Advanced Technology data collectors conducted a series of 
manual edits for the 275 institutions participating in the Campus-Based programs that 
were sampled for the Spring 198<* Title IV Quality Control data collection.* 

'if 

ThcJse edits 'included cross-year comparisons of FISAP data elements and cross- -» 
data source comparisons with the Pell Grant Disbursment System file and Higher 
Education General Information Survey (HEGIS) files. Institutions that -were found to" 
have discrepancies in FISAP data elements were identified -for further data retrieval. 
The results of this first activity are contained in a.report entitled "Analysis of Error 
Associated with the Application' and Allocation Aspects of the Cam pus- Based Pro- 
grams:. Results from Initial Collection,'" submitted to DQA in May 198<*. 

The second activity consisted of a supplemental FISAP data collection conducted 
during the Title IV data collection at those institutions identified through the manual 
edits described above. Finaftciai aid 'administrators and registrars were asked to 
resolve extant discrepancies. These data are still being analyzed and a report will be 
produced in the next several months. . 

The focus of this report is the recommendation^ of edits that can be performed on 
the application portion of the FISAP by the' processor including recommended initial 

tolerances for these edit checks. However, since these^toierances have been 

w " ( 

developed through qualitative analytic methods and since data are currently unavail- 
able with which to establish tolerances through quantative methods, the report also " 
proposes a longer-term approa'ch to assessing the efficacy of recomm^ded edit 
checks, the adequacy of recommended tolerances. This approach atJjpRludes"} 
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developing a data gatheririg. and analysis plan for analyzing -the validity of potential 
edits and reasonable tolerances forffiese edits^x w 



Background apd -Nature of the Problem 



/ 



The application and funding! jrfB^* or three Campus-Based ^ograrns, 
College Work-Study (CW-5), Nationafl^HK Student Loan (NDSL), and Supplemental 
EducationaVopportunity Grants "(SEOG) is complex, time-consuming, and potentially' 
error prone. Campus-Based student financial aid funds are allocated to participating 
institutions in each state by a process in which the overall financial need of students 
at one eligible institution is compared to the need of students attending other 
applicant institutions in that state and in the nation as a whole. For £oth the ND^L 
and CW-5 program's, an institution receives" an allocation computed in three^ general, 
stages from data filed by institutions on the application portion of the FISAP: 
• * " * 

(1) A'conditional guarantee 

(2) A, state increastfoased on its "fair shaqe" oj the state apportionment 

(3) , A national increase based on its "fair share" of the national appropriation. 

For SEOG, the^ allocation is computed in four general stages (for previous program 
participants): v r . ■ . 

* 

(1) A conditional guarantee 

(2) An initial year (IY) state increase based on its "fair sha^e" .of the state- s IY 
apportionment * 



r/pon 



(3) An IY national increase, based on its "fair share" of the -national IY/portion 
of the a $EOG appropriation 

(4) A continuing year (CY) national increase based on its "fair share"' of the 
national C Y portion of the SEOG appropriation. 

Through more than a decade of evolution the attempts to increase the validity of 
application forms aVid ensure more accurate and more equitable "distribution of funds 
have complicated the application as well as application processing. The potential f or 
errors by the institutions and the processing system has also increased. This increase 
in complexity *nd resultant increases in the opportunity for error, combined with an 
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increasing number of participating institutions (currently over 4,000), have caused a 

* . ■ 

good deal of. attention to be focused on FISAP processing byOSFA and others. 

* * * , , 

This report focuses on ope area that continues to receive attention, namely, the 
accuracy oi data submitted by institutions on *he application portion of the FISAP. 
These* data are cwflcai since they determine, to a degree, the institutions* funding 

ft 

level fojr ; eacfi o£ the Campus-jaased programs. 

r ■ • 

Application -Processing . 

- . " ' -'" - - . 

A major area ofr OSIj^K concern relates to the computerized processing of the 
FISAP by the contractor which produces an. institution's initial allocation. The 
allocation is based on the funding formula, current regulations,' and the program, 
appropriations. Last-minute " changes to regulations - and • delays in receiving the 
appropriations- result in last-minute modifications tp program modules inVthe process- 
ing system. Edit checks must be modified to accomodate these changes,^potentiaily 
resulting in processing delays. While such modifications cannot be controlled, they, 
must be considered in developing an approach to systematic application edit checks. 

Ideally, institutions would like to know in January how much money will be 
available to award to students for the following academic year.- In recent years many 
institutions have not received a final allocation before awarding aid to students. In 
part this delay^has been caused by the last-minute nature of the changes noted above 1 
and delays in the funding process. However, when considering a comprehensive system 
of edit checks, care must be taken in. order not to overburden a large number of 
institutions with error messages and correction requests which would slow corrections 
and processing required to produce the final, allocation. Such edits can quickly reach 
the*j5oint of- diminishing return. v// 

■v. 

Foiiow-up Procedures 

* 

Once schools have received error messages noting specific line items on the 
FISAP which failed to pass system edit checks, it is their responsibility to make the 
necessary corrections and forward the corrected items to OSFA for reprocessing. 
Although certain uncorrected items will trigger a flag in the system and the institution 
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'be put ■ on; "hoW, u bther, correction failures" go through the system and still permit 
'. <an ■iteW&iob _ toi'' receive an 'initial award /It^sonfte cases that initial, allocation is in 
; error ^any^.tt^ii^tiiution unknowingly ''^ards funds :lt is not entitled, to award. 
; Jnabljityo of MsF-A to follow up the status of correction's and. appeals to initial 
^allocations is due in large part ^insufficient staff to track corrections and appeals 
* _wl^c^ : ^^ally come^rom fn'inejf^aiing-pefcentage^f over <*,0Q0 schools participating 
^ii^he^to^am. System tracking' procedures for corrections submitted for reprocess-" 
te^.>^giftg .reports) and prompt resolution of allocations for "hold" institutions are 
.Syjp^oKt^ ^ollow-yg "procedures which . are , contractor responsibilities. Each of these 
^ar^as affects' the^uality and integrity of the $SAP process and is,^ another aspect of 
\osfa concern. This report deals in a limited manner with these areas. 



GENERAL APPROACH TO DEVELOPING EPIT CHECKS 



,*2# The'- Campus^a^'ed' processor's draft quality control plan identifies 109 edit 

checks ';f or the- FISAP. However, only 13 of these edits relate to the application 

portion of the FISAP. In addition, these edits are restricted primarily to internal. 

I * 
arithmetic and consistency tests. , 

The edit checks recommended in this report include several additional internal 
. , -r 
edits.- Also recommended are cross-year edits checks and cross-data source edijs 

checks using Pell and HEGIS data. .'*'"* 



Fundamental Design Principles 



The; development of the series of edit checks enumerated in the hrftowing 
•sections is based on sevfral principles which focus on the inherent qualities of an 
effective quality control system for the application portion of FISAP and its 
relationship to the Campus-Based delivery system, particularly the capabilities of the 
two main actors, institutions, and ED. The principles embodied in our approach 
include: - f 



• ; Comprehensiveness 

m 

m Efficiency 

• Congruity 



BEST COPY AVAILABLE 



• - VerifiabUity; • 

Comprehensiveness as a design principle ensures that all meaningful edits will be 
conducted and ail possible data sources -will be utilized. Efficiency has more 
dimensions as a design principle. The first dimension considersxthe capability of the 
system to discriminate between erroneous data and reasonable year to year changes, in 
data that of ten occur at institutions subject, to dynamic circumstances. A system of 
edit checks must.be'. able to identify large numbers of errors in a systematic rather 
than random fashion and to target error prone cases. 

The second dimension of the efficiency principle is the ability to elicit changed 
• in 'data elements from institutions and to track and incorporate these changes into the 
processing system expeditiously. Consequently, neither an inordinately large number 
of edits nor an unnecessarily large number of cases, as discussed above, should be 
produced. Either .condition would have adverse impacts on institutions and the ED 
staff and processor. 

The next design principle is congruity, which implies that the treatment assigned" 
relates directly to the importance of the data item and the reliability of the data 
sources and tolerance levels used in the edits. The last design principle is to maximize 
the' number of edits that have directly verifiable information. 

\ " " . ' • • 

Approach < 

A two*staged apjproach to developing an effective set of edit checks is proposed. 
This approaches proposed for two reasons. First, as our prior reports indicated, there 
is no clear definition - or determination of what constitutes an error. Second, a 
preliminary analysis of pur data collection at institutions raised significant questions 

about direct comparability of certain types of comparison data. This would suggest 

* « 

that more detailed analysis is required. 

{. 

The first stage consists of a thorough review of the current edits and the 
application document in order to develop, evaluate, and recommend edit checks for 
implementation for this processing year. Potential edits were also identified, but are 
not recommended for implementation this year. Rather they are recommended for 
further analysis and implementation during the next processing year. Initial tolerance 

'• 

* - . ' > .• * BEST COPY AVAILABLE 

. ■.- . 28 



■ levels are recommended for this processing year. These were developed through 
qualitative analysis, -including analysis of edit tolerance structures and levels used in 
- other processing systems. First stage activities included the following: 

• Review processor's draft QC plan 

• Analyze othetj processing systems . 

• Analyze proposed edits and the application document . 

* 

« Reyiew edits proposed in the QC plan, as well as omissions , 
r • , Develop comprehensive list of potential edits 

■ • Identify data sources ' 

• Develop initial tolerances " 

• Identify edits to recommend for further analysis and possible implementa- 
tion during 1984-8.5 processing year. 

• ' ' ' . 

The results of this phase are, presented in the next section. Briefly, the results; 
include additional internal edits not covered in the QC plajij internal or external data 
sources for each of the 2<f edits developed? and a tolerance structure that is'sensitive 
both to magnitude of the change in data elements and the reliability of the data 
sources used for comparison.. 

The second phase of the proposed approach has two objectives and ^achieves these 
through developing an analysis plan and conducting longer term analysis. The first of 
these objectives is to refine the tolerance levels based on current processing year 
performance and cross year analysis. Measures of performance could include the rate 
of edit failure, magnitudes and frequencies of corrections. 

The second objective concerns the collection and analysis. o|- data with which to 
assess the feasibility of edits proposed in the following section but not recommended 
for implementation during this processing year. In addition, analysis will be conducted 
to identify additional edits or more elegant and efficient means of conducting edits. 
The approaches that -will be investigated include error prone selection and multi-level 
edits. ' Multi-level edits could include sequencing" edits so that an application that fails 
an initial edit would be subject to a battery of other edits to attempt to determine 

a 



whether the data are the result of legitimate changes in characteristics or actual 
errors. The feasibility, of multi-level treatment for edit failures must also receive 
careful attention. For 'instance, small changes, even though they are suspected to be 
in error can simply be flagged and process may continue. However, errors of large 
magnitudes or patterns may require putting an application on hold. ' . «*i «.V 

We propose to submit such an analysis plan in 60 to £0 days. 
DESCRIPTION OF POTENTIAL FISAP EDITS 



The optimum machine editing system for processing the application portion of 
y 

the FISAP should perform three functions; The system first must ensure that data 
reported on the .FISAP is internally consistent. Checks should be made t<J make 
certain that the respondent's arithmetic is correct, that ail required entries have been 
made, and that the relationships between data are logical. Second, since institutions 
annually submit the. FISAP, the editing system should ensure the cross-year consist- 
ency of data. Most data elements on the FISAP should not change dramatically from, 
year-to-year. Inasmuch as the prior year FISAP files can be considered "clean," they 
$ should be considered as a good base for evaluating values on the current year FISAP. 
Finally, the optimum editing system should ensure the consistency of FISAP data with 
" other higher education, data collected by the Department. of Education. The Pell Grant 
Disbursement System and HEGIS, for instance, collect some data which are defiftitio'n- 
, ally consistent with data collected on the FISAP. < % $ 

* ■ 1 - • 1 - m ' 

Figure i shows the. 13 edit checks currently used on the application portiory of 
the FISAP. As the figure shows, the current edits address only the internal 
consistency of FISAP data. Xs such, the current editing system is limited in the kind 
of errors and inconsistencies it can detect. A FISAP form, for example, may have 
data which is internally consistent—and, therefore, Judged "clean" by the current 
system—yet have error s which have a direct and possibly significant impact on the 
distribution of Campus-Based aid. 

Experience conducting manual validation of FISAPs from 275 sampled institu- 
tions has suggested 24 specific edit checks *that should be considered for ^corporation 
in the FISAP application editing system. The 2U- edits which are specified in Figure 2 
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Institutions with expended . funds in the fiscal- operations report year must 
designate a request for' funds for the upcoming award year on page ii, section A, 
lines i, 2, 3, 4, or 5. 



v. 



Total expenditures on page ii, line 13 must equal the sum of the individual 
expenditures, in lines 10, ii, and 12. 

The NDSL-FCC request, page U, section A, line 2 must be less than the NDSL 
LOE. request, page H, section A, line 1. , ■ 

When tuition revenue is on page 13, section E, line 15, column a, enrollment data 
must be entered on section G, line 36, column a, or lines 39 through 51, column 

. / 

When tuition revenue is on page i>, section E, line 15, column b, enrollment data 
must be entered on section G, line 37, column b, or lines 39 through 51, column 
b. - - 

Total dependent undergraduate eligible aid applicants on page 13, section F, line 
35, column a, must equal-the sum of lines 19 through 34, column a. 

Total dependent graduate/professional eligible aid applicants on page 13, section ' 
F, line 35, colunyi b must equal the sum of lines 19 through 34, column b. 

Total independent undergraduate eligible aid applications on page 13, section F,- 
iine 35, column c, must equal jhe sum of lines 19 through 34, column c. 

Total independent graduate/professional eligible aid applicants on page 13, 
section F, line 35, columned must equal the sum'of lines 19 through 34, column d. 

When total' number of students is entered on page 13, section G, line 36, columns 
a or b, there must.be no entries in Hnes 39 through 51, columns a and b. 

When page 13* section G, has entries shown in lines 39 through 51, there must be 
no entries shown in section F, lines 19 through 35, columns b and d, and in 
section G, line 36, column b. g 

Total n5n-traditional continuing students on page 13, section G, line 51, column a 
must equal the sum of lines 39 through 50, column a. 

Total non-traditional new starts on page 13, section G, line 51, column b must 
equal the sum of iines 39 through 50, column b. 



FIGURE 1 

^DESCRIPTION OF CURRENT EDIT CHECKS 
ON THE APPLICATION PORTION 
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A. INTERNAL CONSISTENCY CHECKS 



Data Elements 
1/ • Pel! Grant Expenditures 

Undergraduate Dependent Applicants 
« Undergraduate Independent Applicants 



Section an4 
Line Number 
References 

EL6 
F35 
F35 « 



Test/Error Condition 

if Peil Grant expenditures were reported in E16 then 
data for dependent or independent undergraduates must 
be reported in F35* 



2. 



• Undergraduate Dependent Applicants 

• Undergraduate Independent Applicants 

• Undergraduate Enrollment — Traditional 

• . Continuing Enrollment— Nontraditionai 

• New Enrollment — Nontraditionai 



F35 
F35 
G36a 
G5la 
G5ib 



The sum of dependent and independent undergraduates 
reported in F35 must be less than or equal to the under- 
graduate enrollment reported in G36a or the sum of con- 
tinuing and new enrollment reported in G51a and G51b. 



• Graduate Dependent Applicants 

• .Graduate independent Applicants 

• Graduate Enrollment 



F35.. 

"F35 

G36b 



The sum of dependent and independent graduates/profes- 
sionals reported in F35*must be less than or equal to 
the graduate/professional enrollment reported in G36b. 
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FIGURE 2 

SPECIFICATIONS FOR ADDITIONAL 
MACHINE EDITS FOR THE 
CAMPUS-BASflfc* SYSTEM 
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B>. CROSS-YEAR CONSISTENCY CHECI 

. A- 

Data Elements 

Institutional Grant Expenditures for 
Award Year 1977-78 

Undergraduate Tuition and Fees 



/ 

Graduate Tuition and Fees 
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) 



Section and 
•Line Number 



References * / ^ * Test/Error Condition " ✓ " 

El 8 . / Institutional grant expenditures must equal correspond- 



/ 



ing data-froiu last year's form. - 



■ El 5a Current fear tqtxt* undergraduate ( tuitio^and fees will 

v . be compared wfttycor responding data from last year's* k %> 

form. The current year data must be within the follow- ' 
y 'ing tolerances which vary according to the level of last 

year's data: ' 

Level of Last - Tolerance Ranges 

r Year's Data for This Year's Data 

$ 0 - $ <*99,999 + 60% of last year's data 

500,000 - 999,999 7 50% of last year's data 

1,000,000 - 9,999,999 + 40% of last year's data 

10,000,000 - 19,999,999 7 30% of last year's data 

20,000,000 and above + 20% of last year's data 

! El 5b Current year total graduate tuition and fees when com- 

pared with corresponding data from last year's form 
' .must fall within the following tolerance ranges: 

Level of Last Tolerance Ranges 

\ ^ Year's Data . for This Year's Data 

^ $ "0 - $ 499,999 + 60% of last year's data 

500,000 -, . 999,-999 7 50% of last year's data" 

.1,000,000 - 9,999,999 7 40% of last year's data 

10 JOO.000 - 19,999,999 w 4 30% of last year's data 
20,T)00,OQp and above 7 20% of last year's data . m 
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B. CROSS-Y EAR CONSISTENCY CHECKS (Continued) 



Data Elements 
Pell Grant Expenditures 



\ 



Section arid 
Line Number 
-References - 

F.16 



> 

State Grant Expenditures 



Ei7 




Test/Error Condition 

-* -> . * 

Current year Peli Grant expenditures when compared 
with corresponding data from last year's form must fail 
within the following tolerance ranges: 



Level of. Last 
year's Data 

$ ' 0 - $ 99,999 
100,000 - 499,999 
500,000 - 999,999 
1,000,000**- 4,999,399 
5,000,000 and above 



v Tolerance Ranges - 
for This Year's' Data 

+_ 60% of last year's data 
+ 50% of list year's data 
+ 40% of last year's dat/v 
_+ 30% of last year's data 
7 20% of last year's data 



Current year state grant expenditures when compared 
with corresponding data from last year's form must fall 
within the following tolerance ranges: 



Level of Last 
Year's Data 



100,000 
500,000 



0 - $ 



99,999 
,499,999 
juv,uuu - 999,999 
1,000,000 - 4,999,999 
5 j 000^ 000 and above 



Tolerance Ranges 
for This Year's Data 

+ 60% of last year's dat;, 
7 50% of last year's dati . 
,7 40% of last year's dati 
+ 30% of last year's dati 
4 20% ©J last year's, daii— 



. V 
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B. CROSS-YEAR CONSISTENCY CHECKS (Continued) 



Data Elements 

• Undergraduate Dependent Applicants 

• Undergraduate Independent Applicants 



Section and 
Line Number 
. References 

1 F35 

F35 



Test/Error Condition 



The sum of dependent and independent undergraduates for 
the current year' when compared with corresponding data 
from last year's form must fail within the following 
tolerance ranges: 



Level of Last - 
Year's Data 

0 - 99 

100 - 499 

•500 - 999 

1,000 - <f,999 

3,000 and above 



Tolerance Ranges 
for This Year** Data 

+ 60% of last year's data 
+ 50% of last year's data 
jf 40% of last year's data 
♦ 30% of last year's data 
+ 20% of last year's data 



• v Graduate Dependent Applicants 

t 

• Graduate Independent Applicants 



F35 



F35 



The sum of dependent and independent graduates for the 
current year when compared with corresponding data 
from last year's form must fall within the following toi- / 
erance ranges: 



Level of Last 
Year's Data 



100 < 



99 
<f99 

500 - 999 
1,000 - 4,999 
5 , 000 and above 



Tolerance Ranges 
for This Year's Data 

± 60% of last year's data 
j_ 50% of last year's data 
+_ 40% of last year's data 
*_ 30% of last year's data 
♦ 20% of last year's data 
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B. CROSS-YEAR CONSISTENCY CHECKS (Continued) 



Data Elements 



Undergraduate Enrollment -~ Traditional 



Section and 
Line Number 
.References 

G36a 



Test/Error Condition , 

' Current year undergraduate entailment when compared 
with corresponding data from last year's form must fail 
within the following tolerance ranges: 



Level of Last 
Year's Data 

* 

0 - 99 
100 - 499 
500 - 999 
1,000 - ' 4,999 
5,000 and above 



Tolerance Ranges 
for This Year's Data 

+ 60% of last year's data 
+ 50% of last year's data 
♦ 40% of last gear's data 
+ 30% of last year's data 
+ 20% of Ja*t year's data. 



Graduate Enrollment 



G36b 



Current year graduate/professional enrollment when' 
compared with corresponding data from last year's must 
fall within the following tolerance ranges: 



Level of Last 
Year's Data ____ 

0 - J 99 

100 - 499 

500 - 999 

1,000 - 4,999 

5,000 and above 



Tolerance Ranges 
for This Year's Data 

± 60% of last- year's data 
7. 50% of last year's data 
7 40% of last year's data 
7 30% of last year's data 
7 20% of last year's data 
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B. CROSS-YEAR CONSISTENCY CHECKS (Continued) 



„ Data Elements 
10. • Continuing Enrollment ~ Nontraditional 
• New Enrollment — Nontraditional 



lY^ • Total Undergraduate Tuition and Fees. 

• Undergraduate Enrollment — Traditional 

• Continuing Enrollment — Nontraditional 
m New Enrollment — Nontraditional 



i 

J. 



Section and 
Line Number 
, References 

*G51b 



El 5a 
G36a 
G51a 

» 

G5ib 



Test/Error Condition 

The sum of continuing and new enrollment for the cur- 
rent year when compared with corresponding data from 
last year's form .must fail witliin the following tolerance 
ranges: . . ' 



Level ol Last 
Year's Data 



0 

100 
500 
1,000 
5,000 



99 
'499 
999 
4,999 
and above 



Tolerance Ranges < * 
for Tlus,j Year's Data 

■ i 

. j_ 60% of last year's data. 

. 7*50% of last year's data 
7 40% of last year's data, 
7 30%4of last year's data 
> 20% of last year's data 



if total undergraduate tuition and fees increase by more 
than 10% when compared with last year's data, undergrad 
uate enrollment in traditional institutions or the sum of 
continuing and new enrollment in nontraditional schools 
must not decrease by 10% Or more. 



12. 



To 1*1 Graduate Tuition and Fees 



9- Graduate Unroliment 



El 5b 
G36b 



If total graduate tuition and fees increase by more than 
10% when compared with last year's data, graduate en- 
rollment must not decrease by 10% or more. 



9 

.ERJC 
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B. CROSS-YEAR CONSISTENCY CHECKS (Continued) 



Data Elements - 
13. • Total Undergraduate. Tuition and Fees 

• Undergraduate Enrollment — Traditional 

• Continuing Enrollment — Nontraditional 

• „New Enrollment — Nontraditional 



Section and 
Line Number 
References 

El 5a 

G36a 

G51a 

G5ib 



Testy Error Condition 

If total undergraduate tuition and fees decrease by 
more than 10% when compared witty last year's data, 
undergraduate enrollment in traditional institutions or 
the sum of continuing and new enrollment in nontradi- 
tional schools must not increase by 10% or more. 



14. • Total Graduate Tuition and Fees 

< 

• Graduate Enrollment \ , 



E15b 
G36b 



If total graduate tuition and fee* decrease by more than 
10% when compared with last year's data, graduate enroll- 
ment must not increase by 10% or more. 



15. • Pell Grant-Expenditures 

• Dependent Undergraduate Applicants 
wi'lh/ncomes From $0 - $26,999 

• Independent Undergraduate Applicants 
with Incomes From $0 - $8,999 



EI6 

Fi9 through F27 
F19 through F27 



If Pell Grant expenditures increase by 10% or more when 
compared with last year's data, the sum of dependent 
applicants with incomes below $26,999 and independent 
applicants with incomes belo^J>8,999 must not decrease 
by 10% or more. 



16. 



ERJC 



Pell Grant Expenditures 

Dependent Undergraduate Applicants 
with Incomes From $0 - $26,999 

Independent Undergraduate Applicants 
with Income From $0 - $8,999 
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E16 

F19 through F27 
F19 through* F27 



If Pell Grant expenditures decrease by 10% or more 
when compared with last year's data, the sum of depen- 
dent applicants with incomes below $26,999 and indepen- 
dent applicants with incomes below $8,999 must not 
increase by 10% or more. 
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C. CONSISTENCY CHECKS WITH OTHER DATA BASES 



Data Elements 
4 Total Undergraduate Tuition and Fe^js 
• Total Graduate Tuition and Fees 

Pelt Grant Expenditures 



• Undergraduate Enrollment ~ Traditional 

• Graduate Enrollment 



■ Section and 
Line Number 
. References 

El5a 

El5b 



E16 



G36a 
G36b 



Test/Error Condition 

* * 

The sum of total tuition and fees for undergraduates and 
graduates must not differ, by plus or minus 10% when com 
pared with comparable data on Part A, Line 1 of 11 EG IS' 
Financial Statistics Survey. . ' * 

Pell Grant expenditures must not differ by plus or minus 
10% when compared against comparable data on the Pell 
Grant Disbursement System's Universe File. 



The sum of undergraduate and graduate enrollment must 
not differ by plus or minus 10% when compared with the 
sum of lines I, 10, 1 1, 15, 24, and 25 from columns 13 
and Ik of HEGlS*s Fall Enrollment Survey. 



• Type of Institution 

• Total Undergraduate Tuition and Fees 

• Undergraduate Enrollment > 



E15a 
•G36a 



If the institution is private, nonprofit, the ratio of under- 
graduate enrollment to total undergraduate tuition and 
fees must not differ by more than plus or minus 25% when 
compared with undergraduate tuition and fees .reported oil 
HEGIS' institutional Characteristics Survey. 



• nType of institution • 

• Total Graduate Tuition and Fees 

• Graduate Enrollment 



El 5b 
G36b 



If the institution is private, nonprofit, the ratio of grad- 
enrollment to total graduate tuition and fees must not 
differ by more lhan plus or minus 25% when compared _ 
with graduate tuition and fees reported on HEGIS* insti- 
tutional Characteristics Survey. 1 
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address ali three edit check functions: additional internal consistency checks, cross- 
year c6mparisons, and comparisons with other higher education data files. Figure 2 
describes each of the proposed edits, provides the data elements which each edit 
addresses, and references the section and line number of the data element on the 
form. As Figure 2 indicates, the proposed edits address only those data items in 
Sections £, F, and G on the application portion of the FISAV which are critical in 
determining an institution's need for Campus-Based aid. * 

Proposed Internal Consistency Checks 

ThrSe edits which would check to make sure that the relationships between 
certain data items are logical are proposed.- The first edit would ensure that when Pell 
• Grant expenditures are. reported in Section E, a figure for total undergraduate eligible 
aid applicants is also reported in Section F. The second two edits would make sure 
Hfcat the number of eligible aid applicants reported in Section F is less than or equal to 
the institution's total Fall enrollment reported in Section G. 

.■)'•■ " 

Recommendation 

*We recommend that OSFA incorporate these three internal consistency checks 
into the FISAP -editing system for the upcoming processing year. The three edits 
address. Pell Grant expenditures, number of aid applicants, and total enrollment, three 
data elements which our field work suggests, are among the most often misreported. 
The three edits, by checking the,, consistency of the three data elements, would 
uncover misreporting that the current system is not capable of identifying. 

Proposed Cross- Year Consistency Checks 

t ; 

Sixteen edits . which check an institution's cross-year reporting consistency are 
proposed. In the first edit, a cross-year comparison is made with not tolerance 
provided. In the next 9 edits, a critical item is compared cross-year using a tolerance 
range. If the values of the two items being compared fail within the tolerance range, 
the item would not be considered- in error. The proposed tolerance ranges are 
expressed as proportions of current year data to last year's 'data. For example, if an 
'^istitution reported 80 for its undergraduate enrollment last year its tolerance would 
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be plus or minus 60 percent. Its tolerance range for this year's data would be 
32 (SO - (.60 x SO) = 32} to 128 (SO + (.60 x SO) = 12S). The tolerances vary depending 
on the amounts reported on last year's form. Ranges are broader for small amounts 
and more restrictive for large amounts since significantly more cross-year variation 
should be expected at small institutions. 

; Extensive quantitative analysis of trends irr" FISAP reporting and the possible 
impact of various tolerance ranges on, the frequency of edit failures was not possible 
due to the limited scope of the task. Instead, the proposed tolerance ranges were 
assigned based on experience^ doing manual cross-year comparisons on 275 samples 
FISAPs and on a review of cross-year edits used by the HEGIS surveys. The HEGIS 
program has been using cross-year comparisons for several years and as a result has 
been able to follow trends and refine the tolerance ranges it uses. 

■»■■ * * ' '. 

i 1 * 

The final six cross-year consistency checks examine the relationship of change in 
jwo or more, data elements. In four of the edits, change in total tuition and fees is 
compared to change in total enrollment (e.g., if total tuition and fees increase by more- 
than 10 percent, then enrollment should not decrease by more than 10 percent). In the 
other two edits, change in total Pell Grant expenditures is compared to change in'the 
total number of low income eligible aid applicants (e.g., if Pell Grant expenditures 
increase by 10 percent or more, the number of low income applicants should not 
decrease by 10 percent or more). A tolerance range of 10 percent has been used in 
these comparisons since the relationship between the d^£a elements is not perfect. 
For instance, it is possible for total Peil Grant expenditures to rise and the number of 
low income applicants to drop due to an increase in tuition and fees or a change* in the 
Family Contribution Schedule. Likewise, enrollment may drop and total tuition and 
fees rise due to an across-the-board increase in tuition. 

Recommendation 

s 

/ 

We recommend that OSFA Incorporate the 16 cross-year checks on a test basis 
during the upcoming year recognizing that the proposed tolerance ranges were 
developed through | qualitative analytic methods and- that additional analysis will be 
needed to refine them. This ^analysis should be conducted on an ongoing basis in order 
that this refinement continue and the tolerance not become outmoded. 
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Proposed Consistency Checks With Other Data Bases , 

» 

r 

Five edits are proposed which would compare critical data elements from the 
FISAP with data collected by HEG1S and the Pell Grant program. Tolerance ranges 
are recommended for the comparisons to account for the following two problems in 
validating FISAP data with other data sources: ' 

• Ongoing changes "to the Pell Universe and HEGIS files. The figure for Peil 
Grant expenditures on the FISAP may not match wnat the Pell. Universe 
File contains for either current authorization or net expenditures because 
of an onging reconciliation -process- that continues for months after the 
award year. 'Also, as part of the editing process the J4EGIS files are 
revised over a period of several months following the original submission of 

. the HEGIS form. 

i 

• Definitional Differences. In two of the proposed edits, an average tuition 
and fees figure would be calculated (total tuition and fees divided by 
enrollment) and compared against a tuition and fees figure reported for 
private institutions on HEGIS' Institutional Characteristics Survey.* That 
survey, however, asks for the modal (most common) tuitiori and fees rather 
than the campus-wide average. Thus, a rather liberal tolerance of 25 
percent is suggested in making the comparison. 

, , « ^ 4 t • 

> ' ■ 

Recommendation 



We recommend that ways of overcoming the problems we identified in the 
manual validation' of 275 sampled forms be explored before incorporating comparisons 
"with outside data bases into the current editing system. These problems, in addition to 
the two comparison problems already noted, include: 

• Difficulty 'of Identifying Institutions.- The HEGIS and FISAP files have no 
common identifier for the institution other than its name. HEGIS uses the 
- FICE code as the numeric identifer for its institutions, while FISAP uses 

; the entity number. 



♦There is not comparable data for total public institution tuition and fees on the 
Institutional Characteristics Survey. Data is collected on that survey for in-state and 
out-of-state tuition and fees for public institutions.. Without data on in-state and out- 
of-state enrollments, it is impossible to calculate a campus-wide average tuition and 
fees figure. . ' 

/ -i ' . 
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Different Reporting Units. The definition of a reporting unit is different 
for FISAP, Pe.il, and HEGIS. A muiti-campus institution, for example, may 
file a single combined .report for FISAP and separate reports for Pel i. 

Different Number of Reporting Units, Not ail institutions who file a FISAP 
file Peli and HEGIS reports. For example, there are many proprietary 
schools who file the FISAP form but not the HEGIS surveys. 

0 
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